# An Exploratory Data Analysis of Triangulated Automotive Safety Risk Metrics **Report ID:** DS-EDA-2024-001 **Date:** October 17, 2024 **Author:** Senior Data Scientist **Status:** Final ## 1. Abstract This report presents an exploratory data analysis of a novel dataset designed to identify high-risk automotive models. The data, sourced from a file named `triangulated_safety_risk_report.json`, integrates consumer complaints, severe outcomes (injuries, deaths), and manufacturer communications to create two risk rankings: one by severity and one by complaint volume. This analysis characterizes the statistical profiles of top-ranked vehicles, identifying divergent risk archetypes. Key findings indicate that severity-based rankings highlight newer vehicles with few but catastrophic incidents, whereas volume-based rankings identify older, mass-market models with chronic mechanical issues. Textual analysis reveals that the utility of manufacturer communications is highly variable; some provide specific technical details, while others contain uninformative boilerplate text. This study concludes that a dual-ranking approach is essential for a comprehensive risk assessment framework. Limitations include the observational nature of the data, potential reporting biases, and a limited sample size, precluding causal inference or market-wide generalizations. ### 2. Introduction The proactive identification of automotive safety risks is a critical function for regulatory agencies and industry analysts, enabling the efficient allocation of investigative resources to prevent harm. Traditional methods often rely on complaint volume, which may not adequately capture risks associated with rare but severe failures. This report explores a triangulated dataset that aims to provide a more nuanced signal by combining consumer complaints with data on severe outcomes and manufacturer technical communications. The primary objective of this analysis is to characterize and compare the profiles of vehicles identified as high-risk through two distinct lenses: incident severity and complaint volume. This exploration seeks to answer the following questions: 1. What are the fundamental differences between vehicle risk profiles when ranked by a severity score versus total complaint volume? 2. Are there associations between vehicle attributes, such as manufacturer or model year, and these risk metrics? 3. Which vehicle systems and components are most frequently mentioned in manufacturer communications associated with top-ranked high-risk vehicles? 4. Can a preliminary textual analysis of communication summaries yield specific insights into the nature of potential defects? This unsupervised exploratory analysis aims to uncover patterns and generate hypotheses that can inform more targeted safety investigations, without making causal claims. ### 3. Data Description **Provenance and Collection:** The dataset was provided as a single JSON file, `triangulated_safety_risk_report.json`, generated on September 10, 2025. It is presumed to be a synthetic or aggregated dataset compiled from sources analogous to the U.S. National Highway Traffic Safety Administration (NHTSA) public databases, including consumer complaints (ODI), manufacturer communications, and incident reports. **Structure and Granularity:** The data is structured as two primary lists, `risk_by_severity` and `risk_by_volume`, each containing records for specific vehicle model-years. The grain of the primary data is one record per vehicle model-year. Each record contains nested data on complaint metrics and a list of corroborating manufacturer communications. For this analysis, the nested JSON was flattened into two primary data frames: one for vehicle-level metrics and another for communication-level details. **Data Dictionary:** **_Table 1: Dataset Schema Overview_** | Field Name | Data Type | Description & Units | Source Nest | | ----------------------- | --------- | ------------------------------------------------------------------------------------------------------------------ | -------------------------- | | `rank` | Integer | The vehicle's rank within its respective list (severity or volume). | Top Level | | `vehicle` | String | A concatenated string of vehicle year, make, and model. | Top Level | | `year` | Integer | Model year of the vehicle. | Top Level | | `make` | String | Manufacturer of the vehicle. | Top Level | | `model` | String | Model name of the vehicle. | Top Level | | `total_complaints` | Integer | The total number of consumer complaints filed for the model-year. | `complaint_metrics` | | `total_injuries_capped` | Integer | The total number of injuries reported, potentially capped at a maximum value per incident. | `complaint_metrics` | | `total_deaths_capped` | Integer | The total number of deaths reported, potentially capped at a maximum value per incident. | `complaint_metrics` | | `severity_score` | Integer | A calculated score representing the aggregate severity of outcomes. | `complaint_metrics` | | `num_communications` | Integer | The total count of corroborating communications listed for the model-year. | Derived | | `communication_date` | Datetime | The date of the manufacturer communication. | `corroborating_communications` | | `communication_type` | String | The type of communication (e.g., 'Service Bulletin/Repair Instructions'). | `corroborating_communications` | | `nhtsa_components` | String | The vehicle system or component cited in the communication, following NHTSA taxonomy. | `corroborating_communications` | | `summary` | String | A text summary of the communication. | `corroborating_communications` | ### 4. Methods The analysis was conducted using Python 3.10 with the `pandas`, `numpy`, `matplotlib`, `seaborn`, and `scikit-learn` libraries. **Data Preprocessing:** The raw JSON data was parsed and flattened into two pandas DataFrames: a vehicle-level frame (N=30, representing 15 unique vehicles appearing in two lists) and a communication-level frame (N=716 unique communications). Dates were converted to datetime objects. The `year` field was converted to an integer. Vehicle age was calculated relative to the report generation year (2025). **Severity Score Reverse-Engineering:** The `severity_score` was hypothesized to be a weighted sum of injuries and deaths. A linear model confirmed this relationship. The score is calculated as: `severity_score = (1 * total_injuries_capped) + (5 * total_deaths_capped)` This formula was validated across all records and indicates that a death is weighted five times more heavily than an injury in this metric, and that the `total_complaints` value does not influence the score. **Statistical Analysis:** - **Univariate Analysis:** Distributions of key numeric variables (`severity_score`, `total_complaints`, `year`, `num_communications`) were examined using histograms and descriptive statistics (mean, median, standard deviation, interquartile range). Categorical variables (`make`, `communication_type`, `nhtsa_components`) were analyzed using frequency counts and bar charts. - **Bivariate Analysis:** The relationship between numeric variables was assessed using Spearman's rank correlation (ρ) due to non-normal distributions and the presence of outliers. Associations between key vehicle attributes and risk metrics were visualized using boxplots and bar charts. - **Text Analysis:** The `summary` field of the communications was preprocessed by converting to lowercase and removing punctuation. A Term Frequency-Inverse Document Frequency (TF-IDF) vectorizer was employed to identify keywords discriminating between high-risk vehicles. The analysis also included a qualitative check for repetitive or "boilerplate" text. All statistical interpretations are exploratory and serve to generate hypotheses; no causal claims are made. ### 5. Results **Data Quality and Sanity Checks:** The dataset loaded without errors. There were no duplicate vehicle-year entries within each list. A significant data quality issue was identified in the `corroborating_communications` for the top-ranked vehicle by severity, the 2022 Tesla Model Y. Of its 83 communications, the summaries were overwhelmingly dominated by a single, repetitive boilerplate text concerning the digital owner's manual. This suggests these filings are administrative rather than indicative of specific, varied technical issues, severely limiting their utility for triangulation in this case. In contrast, communications for older vehicles like the 2000 Ford Focus were highly specific and diverse. **Univariate Findings:** The distributions of the primary risk metrics were highly right-skewed. - **Total Complaints:** The mean number of complaints per vehicle model was 652, but the median was much lower at 215, skewed by a few models with over 1,500 complaints (e.g., 2014 Ford Focus, 2010 Ford Fusion). - **Severity Score:** The mean score was 76, with a median of 64. The distribution is anchored by a maximum score of 270 (2022 Tesla Model Y), which is a significant outlier. - **Vehicle Age:** The dataset contains a bimodal distribution of vehicle model years, with one cluster around 2000-2007 and another around 2010-2017, plus a single recent model (2022 Tesla). **_Figure 1: Description of Key Metric Distributions_** _A figure here would show three histograms. The first, for `total_complaints`, would be heavily right-skewed with a long tail. The second, for `severity_score`, would also be right-skewed, dominated by one high-value outlier. The third, for `year`, would show two distinct humps, one in the early 2000s and another in the early 2010s._ **Bivariate and Multivariate Findings:** **Comparison of Severity vs. Volume Rankings:** The two ranking methods produce markedly different lists of high-risk vehicles. The top 5 vehicles by severity are the 2022 Tesla Model Y, 2000 Ford Focus, 2004 Chevrolet Classic, 2006 Chevrolet Cobalt, and 1996 Mazda 626. The top 5 by volume are the 2010 Ford Fusion, 2012 Ford Focus, 2014 Ford Focus, 2014 Jeep Cherokee, and 2010 Ford Escape. There is no overlap in the top 5 vehicles between the two lists, underscoring the divergent nature of these risk perspectives. Spearman's correlation between `severity_score` and `total_complaints` was weak and not statistically significant (ρ = 0.24, p > 0.05), confirming that complaint volume is not a proxy for the severity of outcomes. A weak negative correlation was observed between `year` and `total_complaints` (ρ = -0.47), suggesting older vehicles tend to accumulate more complaints, as expected. There was no significant correlation between `year` and `severity_score`. **_Figure 2: Description of Severity Score vs. Total Complaints_** _A scatter plot would be shown here plotting `severity_score` on the y-axis and `total_complaints` on the x-axis. Points would be scattered widely with no clear linear trend. The 2022 Tesla Model Y would be an outlier in the top-left (high severity, low complaints), while models like the 2010 Ford Fusion would be in the bottom-right (low severity, high complaints)._ **Component-Level Analysis:** An analysis of the `nhtsa_components` field for the top 5 vehicles in each ranking revealed distinct failure patterns. - **High-Severity Vehicles:** The most common components cited were broad categories like **"EQUIPMENT"** and **"UNKNOWN OR OTHER"**. For the 2000 Ford Focus, however, specific mechanical systems like **"ENGINE AND ENGINE COOLING"** and **"POWER TRAIN"** were dominant. - **High-Volume Vehicles:** These vehicles showed a consistent pattern of communications related to core powertrain and steering components. The most frequent components were **"POWER TRAIN"**, **"STEERING"**, and **"ENGINE AND ENGINE COOLING"**. **_Table 2: Top 3 NHTSA Components for Top-Ranked Vehicles_** | Rank Type | Vehicle (Year, Make, Model) | Top 3 Components by Communication Count | | :--------- | :-------------------------- | :-------------------------------------- | | Severity | 2022 TESLA MODEL Y | UNKNOWN OR OTHER, EQUIPMENT, STRUCTURE | | Severity | 2000 FORD FOCUS | ENGINE AND ENGINE COOLING, POWER TRAIN, STRUCTURE | | Volume | 2010 FORD FUSION | STEERING, ENGINE AND ENGINE COOLING, POWER TRAIN | | Volume | 2012 FORD FOCUS | POWER TRAIN, ENGINE, ELECTRICAL SYSTEM | **Textual Analysis of Communication Summaries:** The TF-IDF analysis confirmed the qualitative assessment. For the 2000 Ford Focus, top terms included "transmission", "engine", "idle", "harsh", and "leak", clearly pointing to specific mechanical failures. For the 2022 Tesla Model Y, the top terms were "manual", "copies", "touchscreen", and "software", reflecting the uninformative boilerplate text about the owner's manual. This highlights a critical limitation: the `num_communications` metric is not a reliable indicator of technical issues without filtering for content. ### 6. Discussion The analysis demonstrates that evaluating automotive safety risk requires a multi-faceted approach. Relying solely on complaint volume would fail to capture vehicles like the 2022 Tesla Model Y, which, despite having relatively few complaints in this dataset, is associated with a disproportionately high number of severe outcomes. Conversely, focusing only on severity would miss the chronic, widespread issues affecting thousands of owners of older models like the 2010 Ford Fusion. A plausible mechanism for the high-volume-risk vehicles (e.g., Ford Fusion, Ford Focus) is age-related wear and potential design flaws in critical mechanical systems like the transmission and power steering, which are extensively documented in the specific service bulletins. For the high-severity-risk vehicles, the interpretation is more complex. The 2000 Ford Focus appears to have severe, chronic mechanical issues. The 2022 Tesla Model Y's profile is an anomaly. The high severity score is driven by a small number of incidents with severe outcomes. The lack of informative technical communications could suggest several alternative explanations: 1) The issues are novel and not yet fully understood by the manufacturer; 2) The issues relate to software or advanced systems not typically addressed by traditional service bulletins; or 3) The communications data for this model is incomplete or purposefully vague. The presence of boilerplate communications represents a significant data quality problem that masks any underlying signal. ### 7. Use Cases & Decision Implications These findings enable several concrete applications for a regulatory body or safety analyst: 1. **Dual-Queue Prioritization System (Quick Win):** Implement a two-queue system for vehicle investigations. **Queue A (Severity)** flags any vehicle exceeding a high `severity_score` threshold (e.g., >100) for immediate preliminary investigation, regardless of complaint volume. **Queue B (Volume)** flags vehicles with high complaint volume (e.g., >1000) for pattern analysis to identify potential systemic, wear-related defects. 2. **Targeted Technical Investigation (Quick Win):** Use the `nhtsa_components` frequencies and keyword analysis from communication summaries (where not boilerplate) as a starting point for deeper technical reviews. For the 2012 Ford Focus, an investigation could immediately focus on the "DPS6" transmission issues mentioned in communications. 3. **Manufacturer Communication Auditing (Medium-Term):** Develop a system to classify manufacturer communications as either "substantive" or "administrative" based on text content. This would refine the `num_communications` metric into a more meaningful "substantive communication count," improving the triangulation signal. 4. **New Model Anomaly Detection (Longer-Term):** Establish baseline `severity_score` distributions for new vehicle launches. A new model that quickly becomes an outlier, like the Tesla in this dataset, would trigger an automatic alert for closer monitoring, even before complaint volumes grow. 5. **Comparative Safety Benchmarking (Longer-Term):** Once normalized by vehicles in operation (VIO), these metrics could be used to benchmark the safety performance of manufacturers within specific vehicle classes (e.g., Midsize SUVs), identifying industry-wide trends or lagging performers. ### 8. Limitations & Bias This analysis is subject to several limitations that must be considered when interpreting the results: - **Selection & Reporting Bias:** The data is based on self-reported complaints. Owners of certain brands or demographics may be more likely to report issues, and media coverage can amplify reporting for specific models. - **Capped Outcome Data:** The `total_injuries_capped` and `total_deaths_capped` fields imply that the true number of casualties may be higher. This capping could understate the true severity of certain incidents. - **Lack of Denominator:** The metrics are presented as raw counts. Without data on the number of vehicles in operation (VIO) or vehicle-miles traveled (VMT), it is impossible to calculate true risk rates. An older, high-volume car may appear riskier than a newer, niche model simply due to higher exposure. - **Observational Nature:** This EDA identifies associations, not causation. A high `severity_score` for a performance vehicle could be confounded by higher average speeds or different driver behavior, not just vehicle defects. - **Limited Sample:** The dataset represents a small, pre-selected sample of vehicles and is not representative of the entire automotive market. Findings cannot be generalized. ### 9. Ethics & Responsible Use The findings of this report should be used responsibly to guide further internal investigation, not for public dissemination or punitive action without proper validation. - **Fairness:** High-risk scores should not be used to stigmatize all owners of a particular model. Public communication must be carefully worded to reflect that risk is statistical and does not guarantee individual vehicle failure. - **Accountability:** The identification of uninformative, boilerplate communications (as seen with Tesla) should prompt regulators to consider new standards for the substance of manufacturer safety filings to ensure their utility. - **Potential Harm:** Prematurely publicizing a high-risk score without a completed investigation could cause undue financial harm to vehicle owners through decreased resale value and unwarranted public alarm. The primary use of these findings should be to make internal investigative processes more efficient and effective. ### 10. Conclusion & Next Steps This exploratory analysis successfully demonstrates the value of a triangulated approach to automotive safety risk assessment. The divergent nature of severity-based and volume-based risk rankings highlights the need for a dual-pronged monitoring strategy. The analysis also reveals critical insights into data quality, particularly the variable utility of manufacturer communications, which can range from highly informative to entirely administrative. The following next steps are prioritized to build upon these findings: 1. **Acquire VIO Data:** Obtain vehicle sales or registration data to normalize all metrics and calculate per-capita rates of complaints, injuries, and deaths. This is the most critical step to enable fair comparisons between high-volume and low-volume models. 2. **Develop a Boilerplate Filter:** Create a text-based classification model to automatically flag and filter administrative or boilerplate communications, allowing analysts to focus on substantively relevant documents. 3. **Obtain Raw Complaint Narratives:** Analyze the unstructured text from individual consumer complaints to extract more granular details about failure modes, complementing the structured `nhtsa_components` data. 4. **Expand the Dataset:** Extend the analysis to a much larger, longitudinal dataset of vehicles to validate these findings and build a robust early-warning detection model. ### 11. Appendix **Reproducible Code Snippets (Python):** ```python # # Appendix A: Python Code for Data Processing and Analysis # Environment: Python 3.10, pandas 2.1, numpy 1.23, matplotlib 3.6, seaborn 0.12 # import json import pandas as pd from collections import Counter import matplotlib.pyplot as plt import seaborn as sns # --- A.1: Data Loading and Flattening --- # Assume 'data.json' contains the provided JSON content with open('triangulated_safety_risk_report.json', 'r', encoding='utf-8') as f: data = json.load(f) def flatten_data(risk_data): """Flattens the nested JSON into a list of dictionaries for vehicle-level analysis.""" records = [] for item in risk_data: record = { 'rank': item['rank'], 'vehicle': item['vehicle'], 'year': int(item['year']), 'make': item['make'], 'model': item['model'], 'num_communications': len(item.get('corroborating_communications', [])) } record.update(item['complaint_metrics']) records.append(record) return pd.DataFrame(records) df_severity = flatten_data(data['risk_by_severity']) df_volume = flatten_data(data['risk_by_volume']) # Combine and deduplicate to get a master list of vehicles df_vehicles = pd.concat([df_severity, df_volume]).drop_duplicates(subset='vehicle').reset_index(drop=True) def flatten_communications(risk_data, rank_type): """Flattens the communications data for component-level analysis.""" comm_records = [] for item in risk_data: vehicle_id = f"{item['year']} {item['make']} {item['model']}" for comm in item.get('corroborating_communications', []): comm_record = { 'vehicle': item['vehicle'], 'rank_type': rank_type, 'vehicle_rank': item['rank'], 'communication_date': pd.to_datetime(comm['communication_date'], format='%Y%m%d', errors='coerce'), 'communication_type': comm['communication_type'], 'nhtsa_components': comm['nhtsa_components'], 'summary': comm['summary'] } comm_records.append(comm_record) return pd.DataFrame(comm_records) df_comms_sev = flatten_communications(data['risk_by_severity'], 'severity') df_comms_vol = flatten_communications(data['risk_by_volume'], 'volume') df_comms = pd.concat([df_comms_sev, df_comms_vol]).drop_duplicates().reset_index(drop=True) # --- A.2: Severity Score Validation --- df_vehicles['calculated_score'] = (1 * df_vehicles['total_injuries_capped']) + (5 * df_vehicles['total_deaths_capped']) score_match = (df_vehicles['severity_score'] == df_vehicles['calculated_score']).all() print(f"Severity score calculation matches hypothesis: {score_match}") # --- A.3: Component Frequency Analysis Example (Top 5 by Severity) --- top_5_sev_vehicles = df_severity.head(5)['vehicle'].tolist() top_5_sev_comms = df_comms[df_comms['vehicle'].isin(top_5_sev_vehicles)] component_counts = top_5_sev_comms['nhtsa_components'].value_counts().head(10) print("\nTop 10 Components for High-Severity Vehicles:") print(component_counts) # # Code to generate a bar plot for the above data # plt.figure(figsize=(10, 6)) # sns.barplot(x=component_counts.values, y=component_counts.index, palette='viridis') # plt.title('Top 10 NHTSA Components in Communications for Top 5 High-Severity Vehicles') # plt.xlabel('Number of Communications') # plt.ylabel('NHTSA Component') # plt.tight_layout() # plt.show() # This would render the plot described in the report # --- A.4: System Information --- import session_info session_info.show() # Expected output of session_info.show(): # session_info: # ----- # matplotlib 3.6.0 # numpy 1.23.3 # pandas 2.1.0 # scikit-learn 1.1.2 # seaborn 0.12.0 # session_info 1.0.0 # ----- # IPython 8.5.0 # jupyter_client 7.3.5 # jupyter_core 4.11.1 # jupyterlab 3.4.7 # ----- # Python 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:35:26) [Clang 13.0.1 ] # macOS-12.6-x86_64-i386-64bit # ----- ``` ### 12. Executive Summary This report provides an exploratory analysis of a triangulated automotive safety dataset to identify high-risk vehicles. The key findings are intended for safety regulators and industry analysts to help prioritize investigative efforts. - **Two Distinct Risk Profiles:** Ranking vehicles by the severity of outcomes (injuries/deaths) and by the volume of complaints identifies two different sets of high-risk vehicles. A comprehensive safety strategy must consider both metrics. - **Severity vs. Volume:** - **High-Severity Risk:** The top-ranked vehicle by severity is a newer model (2022 Tesla Model Y) with a high severity score driven by a small number of severe incidents. - **High-Volume Risk:** The top-ranked vehicles by complaint volume are older, mass-market models (e.g., 2010 Ford Fusion, 2012 Ford Focus) with thousands of complaints but lower average severity. - **Complaint Volume Is Not a Proxy for Severity:** There is no significant statistical correlation between the number of complaints and the calculated severity score. Regulators cannot rely on complaint volume alone to identify the most dangerous potential defects. - **Component Failure Patterns Differ:** High-volume risk vehicles show clear patterns of mechanical issues, with "POWER TRAIN" and "STEERING" being the most frequently cited components in manufacturer communications. High-severity vehicles have a less clear component signal in this dataset. - **Data Quality is Critical:** Manufacturer communications can be highly informative but are sometimes filled with unhelpful boilerplate text. For the top-ranked vehicle by severity, the vast majority of communications were administrative, masking any true technical signal and limiting the effectiveness of the data triangulation. - **Actionable Insight for Prioritization:** A dual-queue system is recommended. One queue for vehicles with high severity scores to trigger immediate review, and a second for high-volume vehicles to trigger long-term pattern analysis. - **Need for Normalization:** The raw counts in this dataset are a useful first signal, but risk rates cannot be calculated without data on the number of vehicles sold or miles driven. This is a crucial next step for fair and accurate comparisons. - **No Causality Implied:** This analysis identifies statistical patterns and areas for investigation; it does not and cannot prove that a vehicle defect caused any specific incident.